Context aware midair projection display

ABSTRACT

Projecting midair projections in a 3D space for multiple users, such that the midair projection is not an obstacle to users within the 3D space.

BACKGROUND

The present invention relates to image displays, and more specificallyto projection displays which display images in midair.

A projection display can be created in midair. One example of a systemto create such displays is a combination of a holographic projector andpico-projector. Any digital contents can be projected in the midairprojection display using a system known to the art.

In a “smart house”, many of the devices and appliances in the home areconnected by a network, often wirelessly, so they can talk to each otherand to a home controller. While this term is sometimes used as merely afancy remote control system for devices connected to the “Internet ofThings”, a smart home can be much more, monitoring the activities of itsoccupants and anticipating their needs.

In a smart house, a user might be watching a video, having a face toface discussion with another user at a remote location, or partaking insome other activity. The video display for these activities might beprojected in midair for the user, using the technology referred to aboveor other technologies now known or developed in the future, under thecontrol of a home server computer, which determines where to project themidair display for the convenience of the user.

While watching the midair display, the user might change his focusdirection to some other place. If the midair projection display isprojected near his eye, then it will create an obstacle to the user, andthe midair display might also be an obstacle to some other associatednearby user.

SUMMARY

According to one embodiment of the present invention, a method isdisclosed. The method comprising: receiving an audio data setcorresponding to audio from a set of microphone(s) set up to capturesound in a three dimensional space; receiving a visual data setcorresponding to visual images from a plurality of camera(s) set up tocapture visual images in the three dimensional space; analyzing thevideo data set and the audio data set to detect a plurality of personsin the three dimension space; further analyzing the video data set andthe audio data set to determine, for each person of the plurality ofpersons, a respectively corresponding zone of attention within the threedimensional space; determining a projection zone within the threedimensional space, with the determination being made so that theprojection zone avoids blocking each person of the plurality of personsfrom viewing that person's respectively corresponding zone of attention;and projecting a midair projection display at the projection zone.

According to another embodiment of the present invention, a computerprogram product is disclosed. The computer program product having acomputer comprising at least one processor, one or more memories, one ormore computer readable storage media. The computer program productcomprising a computer readable storage medium having programinstructions embodied therewith. The program instructions executable bythe computer to perform a method comprising: receiving, by the computer,a visual data set corresponding to visual images from a plurality ofcamera(s) set up to capture visual images in the three dimensionalspace; analyzing, by the computer, the video data set and the audio dataset to detect a plurality of persons in the three dimension space;further analyzing, by the computer, the video data set and the audiodata set to determine, for each person of the plurality of persons, arespectively corresponding zone of attention within the threedimensional space; determining, by the computer, a projection zonewithin the three dimensional space, with the determination being made sothat the projection zone avoids blocking each person of the plurality ofpersons from viewing that person's respectively corresponding zone ofattention; and projecting, by the computer, a midair projection displayat the projection zone.

According to another embodiment of the present invention, a computersystem is disclosed. The computer system comprising a computercomprising at least one processor, one or more memories, one or morecomputer readable storage media having program instructions executableby the computer to perform the program instructions. The programinstructions comprising: receiving, by the computer, a visual data setcorresponding to visual images from a plurality of camera(s) set up tocapture visual images in the three dimensional space; analyzing, by thecomputer, the video data set and the audio data set to detect aplurality of persons in the three dimension space; further analyzing, bythe computer, the video data set and the audio data set to determine,for each person of the plurality of persons, a respectivelycorresponding zone of attention within the three dimensional space;determining, by the computer, a projection zone within the threedimensional space, with the determination being made so that theprojection zone avoids blocking each person of the plurality of personsfrom viewing that person's respectively corresponding zone of attention;and projecting, by the computer, a midair projection display at theprojection zone.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a diagram of a user interacting with a midair display in asmart house.

FIG. 2 shows a diagram of multiple users in the smart house of FIG. 1.

FIG. 3 shows a diagram of multiple users in a smart house, with a midairdisplay.

FIG. 4 shows a flowchart of a method of projecting midair projections ina 3D space for multiple users.

FIG. 5 illustrates internal and external components of a components ofthe “smart house” in which illustrative embodiments may be implemented.

DETAILED DESCRIPTION

FIG. 1 shows an example of a “smart house” 10, in which a single user 20is looking at a display 22 projected in mid-air at a location in thehouse. The display 22 can be projected by a plurality of projectors 12,which may be holographic projectors or pico projectors or the like,which are mounted around the house 10 on the walls or ceiling.Preferably, the projectors 12 are mounted on a thrust bearing and pivotjoint 34, which allows a motor to rotate and change the angle of theprojector 12, thus moving the midair image 22 around the house 10. Thevarious pieces of equipment in the smart house 10 will be controlled bya central server computer 18, which is networked to the various piecesof equipment in the house 10 through a wireless access point 16 and/orwired router or switch 24.

The components of the “smart house”, including, but not limited to thedisplay 22, projector 12, central server computer 18, wireless accesspoint 16, switch 24 includes a set of internal components 800 a and aset of external components 900 a, further illustrated in FIG. 5.

FIG. 5 illustrates internal and external components of components of the“smart house” 10, including, but not limited to display 22, projector12, central server computer 18, wireless access point 16, switch 24, inwhich illustrative embodiments may be implemented. In FIG. 5, thecomponents include respective sets of internal components 800 a andexternal components 900 a. Each of the sets of internal components 800 aincludes one or more processors 820, one or more computer-readable RAMs822 and one or more computer-readable ROMs 824 on one or more buses 826,and one or more operating systems 828 and one or more computer-readabletangible storage devices 830. The one or more operating systems 828 arestored on one or more of the computer-readable tangible storage devices830 for execution by one or more of the processors 820 via one or moreof the RAMs 822 (which typically include cache memory). In theembodiment illustrated in FIG. 5, each of the computer-readable tangiblestorage devices 830 is a magnetic disk storage device of an internalhard drive. Alternatively, each of the computer-readable tangiblestorage devices 830 is a semiconductor storage device such as ROM 824,EPROM, flash memory or any other computer-readable tangible storagedevice that can store a computer program and digital information.

Each set of internal components 800 a also includes a R/W drive orinterface 832 to read from and write to one or more portablecomputer-readable tangible storage devices 936 such as a CD-ROM, DVD,memory stick, magnetic tape, magnetic disk, optical disk orsemiconductor storage device. Software can be stored on one or more ofthe portable computer-readable tangible storage devices 936, read viaR/W drive or interface 832 and loaded into hard drive 830.

Each set of internal components 800 a also includes a network adapter orinterface 836 such as a TCP/IP adapter card. Software can be downloadedto the components from an external computer via a network (for example,the Internet, a local area network or other, wide area network) andnetwork adapter or interface 836. From the network adapter or interface836, software is loaded into hard drive 830. Software can be downloadedto the server computer 18 from an external computer via a network (forexample, the Internet, a local area network or other, wide area network)and network adapter or interface 836. From the network adapter orinterface 836, software is loaded into hard drive 830. The network maycomprise copper wires, optical fibers, wireless transmission, routers,firewalls, switches, gateway computers and/or edge servers.

Each of the sets of external components 900 a includes a computerdisplay monitor 920, a keyboard 930, and a computer mouse 934. Each ofthe sets of internal components 800 a also includes device drivers 840to interface to computer display monitor 920, keyboard 930 and computermouse 934. The device drivers 840, R/W drive or interface 832 andnetwork adapter or interface 836 comprise hardware and software (storedin storage device 830 and/or ROM 824).

Software can be written in various programming languages includinglow-level, high-level, object-oriented or non object-oriented languages.Alternatively, the functions of software can be implemented in whole orin part by computer circuits and other hardware (not shown).

In the example of FIGS. 1 and 2, these pieces of equipment include acamera 14, a storage device 26 and a source of programming 28 (whichmight be, for example, a video player). The router 24 can also link theserver computer 18 to a wide-area network 30 such as the Internet, andadditional wires or cables 32 can lead to other devices around the house10.

This setup is straightforward if there is only a single viewer, as shownin FIG. 1. The display 22 can float in midair, and the user 20 can watchit. However, if there is more than one user for the system, as shown inFIG. 2, the situation becomes more complicated. In the example of FIG.2, there are now four users in the smart house 10—Arthur 42 a, Bob 42 b,Charlie 42 c and David 42 d. David 42 d is watching a display 46,separate from the other three, but Arthur 42 a, Bob 42 b and Charlie 42c are circled around the midair display 22, interacting with each other.In this situation, the midair display 22 will block the field of viewfrom Arthur 42 a to Bob 42 b, which can be annoying for the two.Moreover, while the display 22 is not blocking Charlie's 42 c view ofone of the other participants, he might find it distracting when he istrying to pay attention to Arthur 42 a or Bob 42 b.

FIG. 3 shows the four users from FIG. 2, annotated to show their fieldsof view. As can be seen, David 42 d has an unobstructed view of display46. Arthur 42 a has a field of view 44 a, which forms a cone outwardlyof his eyes, aimed toward Bob 42 b. Bob 42 b, in turn, has a field ofview 44 b, directed toward Arthur 42 a. Charlie 42 c is looking betweenArthur 42 a and Bob 42 b, and his field of view is shown as cone 44 c.The three users who are interacting with each other are, collectively,looking at a volume in space which is shown as shaded area 48, whichwill be termed the “restricted area”. Projection of an image within therestricted area 48 would be annoying to the users 42 a, 42 b and 42 c,and therefore should be avoided. However, outside of the restricted area48, the smart house 10 is free to create midair display 46 for David 42d, who is not interacting with anyone else. The area outside of therestricted area 48 is referred to as the content allowed area.

Referring to FIGS. 1-3, as discussed above, the method of the inventioncan be summarized as follows:

Using camera 14 and microphone feed analysis, the context analysisengine on the server computer 18 will identify (a) users (for exampleArthur 42 a, Bob 42 b and Charlie 42 c) who are involved in face to facediscussion and their level of interaction with each other; (b) the focuspoint or area of interest for each user (44 a-44 d), and (c) the degreeof attention each user is giving toward some form of content beingprovided. The “level of interaction” is defined as how deeply one personis involved in interacting with another person. The level of interactionis high if an obstacle between the first person and the second personwould create a disturbance to both the first person and the secondperson.

Accordingly the computing system will identify the aggregated restrictedarea 48 in 3D space where the midair projection display should not becreated. By identifying an aggregated restricted area 48 in 3D space,the midair content is not created in a space that will create anobstacle to a user or will disturb another user.

Software will identify the remaining area where the midair projection 46will be created, (content allowed area) so the software will set thedisplay area which is convenient to David 42 d (nearby him in the samefacial direction) and will not affect any other user in the surroundingarea.

Identification of the users and the determining the restricted area maybe repeated at some interval by the computing system while at least onemidair projection is being displayed or users are present within thehome.

For any notifications and/or newly created midair projections, thecomputing system identifies the user(s) which will be watching orinteracting with the newly created midair projection and generates thenotification and/or newly created midair projections in the display areanear the user and not in the aggregated restricted area calculated forall identified users. Alternatively, the notification may sent to amobile device of the user by the computer system.

The user, for which the midair projection is for display to can beidentified based on various content. For example, the user can beidentified based on which user initiated the midair projection, specificrequests sent to media associated with a specific user, for example viaan email address or other identification associated with the user, orbased on user preferences for content.

For example, if a user, David 42 d receives a phone call notification,the computing system places the midair projection associated with thephone call notification and phone call in the display area which isnearby David 42 d and in the same facial direction as David 42 d and isnot present in the aggregated restricted area 48 in three dimensional(3D) space. Alternatively, David 42 d receives a notification regardingthe phone call on his mobile device and the midair projection associatedwith the phone call is projected in the display area which is nearbyDavid 42 d and in the same facial direction as David 42 d and is notpresent in the aggregated restricted area 48 in 3D space.

For example if David 42 d has initiated a midair projection regardinghow hurricanes form, and Charlie 42 c has a preference for watching andinteracting with programming associated with weather, notifications canbe sent to both David 42 d and Charlie 42 c and the midair projectiondisplayed in an area that is not present in the aggregated restrictedarea 48 in 3D space and is in an area of the house that is navigable tofor both David 42 d and Charlie 42 c.

It should be noted that if a plurality of users are interacting with amidair display, the dimensions and shape of the midair projection beingdisplayed to the plurality of users can be altered automatically as theuser's focus point and degree of attention each user is giving towardthe content varies and thus the aggregated restricted area in 3D spacevaries. Small movements of the user will not affect the restricted areain a 3D space.

Referring to FIG. 4, in a first step, the computing system identifiesthe users which are having face to face interactions and theirassociated level of interaction (step 202).

For example, using camera 14 and microphone feed analysis, the contextanalysis engine on the server computer 18 can identify proximity betweenusers, facial direction of users, and based on the direction of focus ofthe users, calculate where a user is looking. A face to face interactioncan be identified as a major overlap (e.g. greater than 40%) between twofacial directions of different users. The level of interaction can bedefined as how deeply one person is involved in interacting with anotherperson. The level of interaction is high if an obstacle between thefirst person and the second person in the major overlap would create adisturbance to both the first person and the second person.

Next, the computing system determines where the identified users arefocusing to determine attention level and associated focus areas (step204).

The attention level may be based on behavior analysis conducted by theserver computer 18 based on facial expression of the identified userscaptured by the camera 14, direction of focus, blinking rate of the userand object of focus. If a user is frequently changing their direction offocus, for example focusing on a display and then focusing on anotherobject frequently, the user's level of focus on the display is above athreshold in comparison to a user which is not changing focus directionfrom the display, which is below the threshold.

The computer system aggregates all focus areas of each identified userto restrict midair projection from the aggregated focus areas of allidentified users and designate all areas not in the aggregated focuscones as content allowed areas where content can be projected withoutdisturbing other identified users (step 206).

The computer system displays at least one midair projection in thecontent allowed area to at least one identifier user (step 208) and themethod ends.

Steps 202 through 206 are repeated at an interval by the computingsystem after the midair projection is displayed and while midairprojections are being displayed or users are present within the areabeing monitored by the cameras. The interval for repeating steps 202-206may be every 30 seconds or may be calculated dynamically based oncontext from the context analysis engine. If the restricted area isrecalculated and requires movement of the midair projection, the midairprojection is moved to accommodate the recalculated restricted area. Themovement of the midair projection is small and will not be noticeable bythe identified users interacting with the midair projection.

In an alternate embodiment, prior to step 208 of the computer systemdisplaying the at least one midair projection in the content allowedarea, the computer system identifies the user(s) which will beinteracting or watching a newly created midair projection (step 220) andthe computer system sends a notification regarding the creation of amidair projection to a mobile device of the user in which the newlycreated midair projection is for or targeted to (step 222). Other userswithin the vicinity may also be sent notifications by the computersystem based on user preferences and context of the midair projection tobe projected.

In addition, the user who received the notification can additionallyshare the notification with other users in a proximity which may beinterested in the content of the midair projection.

Additionally, the angular orientation of the at least one midairprojection can be calculated by the computer system for the identifieduser to display the at least one midair projection in clear manner.

The present invention may be a system, a method, and/or a computerprogram product at any possible technical detail level of integration.The computer program product may include a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, configuration data for integrated circuitry, oreither source code or object code written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Smalltalk, C++, or the like, and procedural programminglanguages, such as the “C” programming language or similar programminglanguages. The computer readable program instructions may executeentirely on the user's computer, partly on the user's computer, as astand-alone software package, partly on the user's computer and partlyon a remote computer or entirely on the remote computer or server. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the Figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A method comprising: receiving an audio data setcorresponding to audio from a set of microphone(s) set up to capturesound in a three dimensional space; receiving a visual data setcorresponding to visual images from a plurality of camera(s) set up tocapture visual images in the three dimensional space; analyzing thevideo data set and the audio data set to detect a plurality of personsin the three dimension space; further analyzing the video data set andthe audio data set to determine, for each person of the plurality ofpersons, a respectively corresponding zone of attention within the threedimensional space; determining a projection zone within the threedimensional space, with the determination being made so that theprojection zone avoids blocking each person of the plurality of personsfrom viewing that person's respectively corresponding zone of attention;and projecting a midair projection display at the projection zone. 2.The method of claim 1, wherein prior to projecting a midair projectiondisplay, further, comprising: identifying any one person interactingwith an already projected midair projection display; and sending anotification regarding creation of another midair projection to a mobiledevice of the identified any one person.
 3. The method of claim 1,wherein the corresponding zone of attention is determined by facialexpression of each person of the plurality of persons, direction offocus, blinking rate of the user and object of focus captured by theplurality of camera(s).
 4. The method of claim 1, wherein receiving theaudio data set corresponding to audio from the set of microphone(s) setup to capture sound in the three dimensional space; receiving the visualdata set corresponding to visual images from the plurality of camera(s)set up to capture visual images in the three dimensional space;analyzing the video data set and the audio data set to detect theplurality of persons in the three dimension space; further analyzing thevideo data set and the audio data set to determine, for each person ofthe plurality of persons, a respectively corresponding zone of attentionwithin the three dimensional space; and determining the projection zonewithin the three dimensional space, with the determination being made sothat the projection zone avoids blocking each person of the plurality ofpersons from viewing that person's respectively corresponding zone ofattention; are repeated at an interval when a midair projection is beingdisplayed to identified users.
 5. A computer program product forprojecting midair projections in a 3D space for multiple users, acomputer comprising at least one processor, one or more memories, one ormore computer readable storage media, the computer program productcomprising a computer readable storage medium having programinstructions embodied therewith, the program instructions executable bythe computer to perform a method comprising: receiving, by the computer,a visual data set corresponding to visual images from a plurality ofcamera(s) set up to capture visual images in the three dimensionalspace; analyzing, by the computer, the video data set and the audio dataset to detect a plurality of persons in the three dimension space;further analyzing, by the computer, the video data set and the audiodata set to determine, for each person of the plurality of persons, arespectively corresponding zone of attention within the threedimensional space; determining, by the computer, a projection zonewithin the three dimensional space, with the determination being made sothat the projection zone avoids blocking each person of the plurality ofpersons from viewing that person's respectively corresponding zone ofattention; and projecting, by the computer, a midair projection displayat the projection zone.
 6. The computer program product of claim 5,wherein prior to projecting a midair projection display, further,comprising: identifying, by the computer, any one person interactingwith an already projected midair projection display; and sending, by thecomputer, a notification regarding creation of another midair projectionto a mobile device of the identified any one person.
 7. The computerprogram product of claim 5, wherein the corresponding zone of attentionis determined by facial expression of each person of the plurality ofpersons, direction of focus, blinking rate of the user and object offocus captured by the plurality of camera(s).
 8. The computer programproduct of claim 5, wherein receiving, by the computer, the audio dataset corresponding to audio from the set of microphone(s) set up tocapture sound in the three dimensional space; receiving, by thecomputer, the visual data set corresponding to visual images from theplurality of camera(s) set up to capture visual images in the threedimensional space; analyzing, by the computer, the video data set andthe audio data set to detect the plurality of persons in the threedimension space; further analyzing, by the computer, the video data setand the audio data set to determine, for each person of the plurality ofpersons, a respectively corresponding zone of attention within the threedimensional space; and determining, by the computer, the projection zonewithin the three dimensional space, with the determination being made sothat the projection zone avoids blocking each person of the plurality ofpersons from viewing that person's respectively corresponding zone ofattention; are repeated at an interval when a midair projection is beingdisplayed to identified users.
 9. A computer system comprising acomputer comprising at least one processor, one or more memories, one ormore computer readable storage media having program instructionsexecutable by the computer to perform the program instructionscomprising: receiving, by the computer, a visual data set correspondingto visual images from a plurality of camera(s) set up to capture visualimages in the three dimensional space; analyzing, by the computer, thevideo data set and the audio data set to detect a plurality of personsin the three dimension space; further analyzing, by the computer, thevideo data set and the audio data set to determine, for each person ofthe plurality of persons, a respectively corresponding zone of attentionwithin the three dimensional space; determining, by the computer, aprojection zone within the three dimensional space, with thedetermination being made so that the projection zone avoids blockingeach person of the plurality of persons from viewing that person'srespectively corresponding zone of attention; and projecting, by thecomputer, a midair projection display at the projection zone.
 10. Thecomputer system of claim 9, wherein prior to projecting a midairprojection display, further, comprising: identifying, by the computer,any one person interacting with an already projected midair projectiondisplay; and sending, by the computer, a notification regarding creationof another midair projection to a mobile device of the identified anyone person.
 11. The computer system of claim 9, wherein thecorresponding zone of attention is determined by facial expression ofeach person of the plurality of persons, direction of focus, blinkingrate of the user and object of focus captured by the plurality ofcamera(s).
 12. The computer system of claim 9, wherein receiving, by thecomputer, the audio data set corresponding to audio from the set ofmicrophone(s) set up to capture sound in the three dimensional space;receiving, by the computer, the visual data set corresponding to visualimages from the plurality of camera(s) set up to capture visual imagesin the three dimensional space; analyzing, by the computer, the videodata set and the audio data set to detect the plurality of persons inthe three dimension space; further analyzing, by the computer, the videodata set and the audio data set to determine, for each person of theplurality of persons, a respectively corresponding zone of attentionwithin the three dimensional space; and determining, by the computer,the projection zone within the three dimensional space, with thedetermination being made so that the projection zone avoids blockingeach person of the plurality of persons from viewing that person'srespectively corresponding zone of attention; are repeated at aninterval when a midair projection is being displayed to identifiedusers.